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Eckerd Rapid Safety Feedback SM (ERSF) Approach 
Draft Outcomes Table for Oklahoma 


Overview 

ERSF Outcomes Table: CT and ME 


Goals 

1. Fidelity 

2. Child Outcomes 

3. Systems Outcomes 

4. Coaching 

5. Process 

6. Cost 

Out¬ 

comes 

a) ERSF is 
implemented reliably 
and consistently (Is 
fidelity at a reasonably 
high level?) 

a) Reduction in number 
of child deaths: 

1. Overall 

2. In highest 
probability cases 

a) After implementation, 
did things improve, not 
improve, no effect, did it 
get worse? 

a) Reduction over time 
in the % of cases 
reviewed that require an 
immediate coaching 
session 

a) Document the 
implementation of 

ERSF 

a) Economic cost of the 
resources used to 
deliver ERSF process 

b) Ability to monitor the 
quality of 

implementation (We 
need examples of what 
the indicators would be 
for this) 

b) Reduction in serious 
injury: 

• Overall 

• In highest 
probability cases 

b) Increase in workers 
feelings of support, 
confidence and 
empowerment 

d) Adequacy of safety 
plans and coaching 
sessions 

b) Identify facilitators 
and barriers to 
implementation 



c) Reduction in repeat 
maltreatment reports 
(verified and/or 
unverified) 

c) Reduction in staff 

turnover 


c) Identify how ERSF 
helped the agency 
identify necessary 
changes in program 
components 




d) Reduction in severity 
of child maltreatment 
(e.g., number of 
children reported for 
“severe" CAN) 
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Overview ERSF Outcomes Table 


Goals 

1. Fidelity 

2. Child Outcomes 

3. Systems Outcomes 

4. Coaching 

5. Process 

6. Cost 

Out¬ 

comes 

a) ERSF is 
implemented reliably 
and consistently (is 
fidelity at a reasonably 
high level?) 

a) Reduction in number 
of child deaths: 

• Overall 

• In highest 
probability cases 

a) After implementation, 
did things improve, not 
improve, no effect, did it 
get worse? 

a) Reduction over time 
in the % of cases 
reviewed that require an 
immediate coaching 
session 

a) Document the 
implementation of 

ERSF 

a) Economic cost of the 
resources used to 
deliver ERSF process 

b) Ability to monitor the 
quality of 

implementation (We 
need examples of what 
the indicators would be 
for this) 

b) Reduction in serious 
injury: 

• Overall 

• In highest 
probability 

cases 

b) Increase in workers 
feelings of support, 
confidence and 

empowerment 

b) Connection between 
case outcomes and 
level/type/extent of 
staff training 

b) Identify facilitators 
and barriers to 
implementation 



c) Reduction in repeat 
maltreatment reports 
(verified and/or 
unverified) 

c) Reduction in staff 
turnover 

c) Number of staffing to 
measure transfer of 
learning 

c) Identify how ERSF 
helped the agency 
identify necessary 
changes in program 
components 



d) Reduction in severity 
of child maltreatment 
(e.g., number of 
children reported for 
“severe” CAN) 

d) Increase in 
communication re: 

cases between staff 

d) Adequacy of safety 
plans and coaching 
sessions 

d) Does the timing of 
the ERSF practices 
affect results? 



e) Reduction in the 
number of children 
presenting to hospitals 
for CAN 

e) Improved ratings on 
the Critical Case 

Practice Assessment 

Tool 


d) Does the timing of 
the ERSF review affect 

results for the 
investigation phase? 




f) Reduction in repeat 
calls to the CPS hotline 
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1. Fidelity 

Fidelity is the extent to which delivery of an intervention adheres to the protocol or program model originally developed. Fidelity 
measurement has increasing significance for evaluation, treatment effectiveness research, and service administration. 

Outcomes 

Data Source 

Issues and Questions 

a) ERSF is implemented 
reliably and consistently (is 
fidelity at a reasonably high 
level?) 

• Supervisor completed fidelity checklist 
(quarterly) 

• On-site Eckerd evaluation of fidelity 
(quarterly) 


b) Ability to monitor the quality 
of implementation (We need 
examples of what the 
indicators would be for this) 

• Supervisor completed fidelity checklist 

• On-site Eckerd evaluation of fidelity 

• ERSF case record reviews, staff 
surveys and/or focus groups 



2. Child Outcomes 

ERSF practices has an impact on child health and well-being. 

Outcomes 

Data Source 

Issues and Questions 

a) Reduction in number of child 
deaths: 

1. Overall 

2. In highest probability 

cases 

• Child death registries 

o OCCY (OK City) 
o ME? 

o CT? 

o AK? 

• SACWIS 

• Mindshare Portal 

• Subgroup of highest probability cases: rationale? 
Definition? 

• Do data sharing agreements need to be modified or 
developed (placing here, but pertains to any/all 
questions)? 

b) Reduction in serious injury: 

1. Overall 

2. In highest probability 

cases 

• Department of Public Health 

• Data from specific hospital(s) 

• ERSF case record reviews 

• Mindshare Portal (?) 

• SACWIS 

• Define “serious injury” 

• Subgroup of highest probability cases: rationale? 

Definition? 

• Do states have administrative data beyond SACWIS? 

c) Reduction in repeat child 

• ERSF case record reviews, staff 
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maltreatment reports 

surveys or focus groups SACWIS 
• Mindshare Portal (?) 


d) Reduction in severity of child 
maltreatment (e.g., number of 
children reported for “severe” 
CAN) 

• Department of Public Health 

• Data from specific hospital(s) 

• Mindshare Portal (?) 

• SACWIS (?) 

• AK does use SDM future risk of 
abuse/neglect: low, medium and high 
- scored by line workers. Would 
guess error rate is high: around 70% 


e) Reduction in the number of 
children presenting to hospitals 
for CAN 

• Department of Public Health 

• Data from specific hospital(s) 


b) Reduction in repeat calls to 
the CPS hotline 

• SACWIS 

• Mindshare Portal 



3. System Outcomes 

ERSF practices has an impact at the systems level. 

Outcomes 

Data Source 

Issues and Questions 

a) After implementation, did 
things improve, not improve, no 
effect, did it get worse? 

• ERSF case record reviews and/or 

focus groups with staff using ERSF 


b) Increase in workers feelings 
of support, confidence and 
empowerment 

• Staff surveys and/or focus groups 


c) Reduction in staff turnover 

• Supplied by State using HR records 
(trend line percentages) 


d) Increase in communication 
re: cases between staff 

• Staff surveys and/or focus groups 



4. Coaching 


Workers and supervisors have applied the practice knowledge and skills learned in ERSF coaching sessions to new and existing cases. 
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Outcomes 

Data Source 

Issues and Questions 

a) Reduction over time in the % 
of cases reviewed that require 
an immediate coaching session 

• Supervisor completed fidelity checklist 

• On-site Eckerd evaluation of fidelity 

• Mindshare Portal 

• ERSF case record reviews, staff 

surveys and/or focus groups 


b) Connection between case 
outcomes and level/type/extent 
of training 

• Supervisor completed fidelity checklist 

• ERSF case record reviews, staff 
surveys and/or focus groups 


c) Number of staffings to 
measure transfer of learning 

[?? Clear?] 

• Supervisor completed fidelity checklist 

• On-site Eckerd evaluation of fidelity 

• ERSF case record reviews, staff 
surveys and/or focus groups 



5. Process 

How ERSF is implemented in each State. 

Outcomes 

Data Source 

Issues and Questions 

a) Document the 

implementation of ERSF 

• ERSF case record reviews, staff 
surveys and/or focus groups 


b) Identify facilitators and 
barriers to implementation 

• ERSF case record reviews, staff 
surveys and/or focus groups 


c) Identify necessary changes 
in program components 

• ERSF case record reviews, staff 
surveys and/or focus groups 


d) Does the timing of the ERSF 
practices affect results? 

• ERSF case record reviews, staff 
surveys and/or focus groups 


e) Improved ratings on the 
Critical Case Practice 

Assessment Tool 

• ERSF Critical Case Practice 

Assessment Tool 



6. Cost 


The cost of ERSF. 






















Outcomes 

Data Source 

Issues and Questions 

a) Economic cost of the 
resources used to deliver 

ERSF process 

• State-provided summary using 

finance records 







